{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "2e33dced-e587-4397-81b3-d6606aa1738a",
      "metadata": {
        "id": "2e33dced-e587-4397-81b3-d6606aa1738a"
      },
      "source": [
        "<a target=\"_blank\" href=\"https://colab.research.google.com/github/qianniucity/llm_notebooks/blob/main/notebooks/together.ipynb\">\n",
        "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>\n",
        "</a>\n",
        "\n",
        "# 使用 Together AI 和 LlamaIndex 构建您自己的 RAG 应用程序\n",
        "\n",
        "本笔记本展示了如何将 \"Together AI \"用作 LLM。Together AI 提供了许多最先进的 LLM 模型。查看完整的 [模型列表](https://docs.together.ai/docs/inference-models)。\n",
        "\n",
        "访问 https://together.ai 并注册以获取 API 密钥。"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5863dde9-84a0-4c33-ad52-cc767442f63f",
      "metadata": {
        "id": "5863dde9-84a0-4c33-ad52-cc767442f63f"
      },
      "source": [
        "## 安装步骤"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "833bdb2b",
      "metadata": {
        "id": "833bdb2b"
      },
      "source": [
        "安装 LlamaIndex 🦙."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "9bbbc106",
      "metadata": {
        "id": "9bbbc106"
      },
      "outputs": [],
      "source": [
        "!pip install llama-index"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ad297f19-998f-4485-aa2f-d67020058b7d",
      "metadata": {
        "id": "ad297f19-998f-4485-aa2f-d67020058b7d"
      },
      "outputs": [],
      "source": [
        "from llama_index.llms import TogetherLLM"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "152ced37-9a42-47be-9a39-4218521f5e72",
      "metadata": {
        "id": "152ced37-9a42-47be-9a39-4218521f5e72"
      },
      "outputs": [],
      "source": [
        "# set api key in env or in llm\n",
        "# import os\n",
        "# os.environ[\"TOGETHER_API_KEY\"] = \"your api key\"\n",
        "\n",
        "llm = TogetherLLM(\n",
        "    model=\"mistralai/Mixtral-8x7B-Instruct-v0.1\", api_key=\"your_api_key\"\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "d61b10bb-e911-47fb-8e84-19828cf224be",
      "metadata": {
        "id": "d61b10bb-e911-47fb-8e84-19828cf224be"
      },
      "outputs": [],
      "source": [
        "resp = llm.complete(\"Who is Paul Graham?\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "3bd14f4e-c245-4384-a471-97e4ddfcb40e",
      "metadata": {
        "id": "3bd14f4e-c245-4384-a471-97e4ddfcb40e",
        "outputId": "418a3885-e4f5-40da-f9cb-b667fe0f2aae"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Paul Graham is a British-born computer scientist, venture capitalist, and essayist. He is best known for co-founding the startup incubator and investment firm, Y Combinator, which has provided funding and support to numerous successful tech startups including Dropbox, Airbnb, and Reddit.\n",
            "\n",
            "Before founding Y Combinator, Graham was a successful entrepreneur himself, having co-founded the company Viaweb in 1995, which was later acquired by Yahoo in 1998. Graham is also known for his essays on startups, technology, and programming, which have been widely read and influential in the tech industry.\n",
            "\n",
            "In addition to his work in the tech industry, Graham has a background in artificial intelligence and computer science, having earned a Ph.D. in computer science from Harvard University. He is also a prolific essayist and has written several books, including \"Hackers & Painters\" and \"The Hundred-Year Lie: How to Prevent Corporate Abuse and Save the World from Its Own Worst Appetites.\"\n"
          ]
        }
      ],
      "source": [
        "print(resp)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "3ba9503c-b440-43c6-a50c-676c79993813",
      "metadata": {
        "id": "3ba9503c-b440-43c6-a50c-676c79993813"
      },
      "source": [
        "#### 使用信息列表调用 \"聊天 \"功能"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ee8a4a55-5680-4dc6-a44c-fc8ad7892f80",
      "metadata": {
        "id": "ee8a4a55-5680-4dc6-a44c-fc8ad7892f80"
      },
      "outputs": [],
      "source": [
        "from llama_index.llms import ChatMessage\n",
        "\n",
        "messages = [\n",
        "    ChatMessage(\n",
        "        role=\"system\", content=\"You are a pirate with a colorful personality\"\n",
        "    ),\n",
        "    ChatMessage(role=\"user\", content=\"What is your name\"),\n",
        "]\n",
        "resp = llm.chat(messages)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "2a9bfe53-d15b-4e75-9d91-8c5d024f4eda",
      "metadata": {
        "id": "2a9bfe53-d15b-4e75-9d91-8c5d024f4eda",
        "outputId": "5afe3a4e-cec0-438d-85ed-046ff0fba7c1"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "assistant: Arr matey, I be known as Captain Redbeard, the fiercest pirate to ever sail the seven seas! My ship, the Crimson Wave, strikes fear into the hearts of all who dare cross our path. With me hearty crew, we plunder and pillage, always seeking treasure and adventure. But don't be mistaken, I be a fair and honorable pirate, as long as ye show me respect and loyalty. Now, what be your name, landlubber?\n"
          ]
        }
      ],
      "source": [
        "print(resp)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "25ad1b00-28fc-4bcd-96c4-d5b35605721a",
      "metadata": {
        "id": "25ad1b00-28fc-4bcd-96c4-d5b35605721a"
      },
      "source": [
        "### Streaming"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "13c641fa-345a-4dce-87c5-ab1f6dcf4757",
      "metadata": {
        "id": "13c641fa-345a-4dce-87c5-ab1f6dcf4757"
      },
      "source": [
        "使用 `stream_complete` 端点"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "06da1ef1-2f6b-497c-847b-62dd2df11491",
      "metadata": {
        "id": "06da1ef1-2f6b-497c-847b-62dd2df11491"
      },
      "outputs": [],
      "source": [
        "response = llm.stream_complete(\"Who is Paul Graham?\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "1b851def-5160-46e5-a30c-5a3ef2356b79",
      "metadata": {
        "id": "1b851def-5160-46e5-a30c-5a3ef2356b79",
        "outputId": "a469f9c9-552f-46be-c87b-c699ce86dcb5"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            " Paul Graham is a British-born computer scientist, entrepreneur, venture capitalist, and essayist. He is best known for co-founding the startup incubator and investment firm, Y Combinator, which has provided funding and support to numerous successful startups including Dropbox, Airbnb, and Reddit.\n",
            "\n",
            "Before founding Y Combinator, Graham was a successful entrepreneur himself, having co-founded the company Viaweb in 1995, which was later acquired by Yahoo in 1998. Graham is also known for his essays on startups, technology, and programming, which have been widely read and influential in the tech industry.\n",
            "\n",
            "In addition to his work in the tech industry, Graham has a background in computer science and artificial intelligence, having earned a PhD in this field from Harvard University. He has also taught programming and entrepreneurship at several universities, including Harvard and Stanford."
          ]
        }
      ],
      "source": [
        "for r in response:\n",
        "    print(r.delta, end=\"\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ca52051d-6b28-49d7-98f5-82e266a1c7a6",
      "metadata": {
        "id": "ca52051d-6b28-49d7-98f5-82e266a1c7a6"
      },
      "source": [
        "使用 `stream_chat` 断点"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "fe553190-52a9-436d-84ae-4dd99a1808f4",
      "metadata": {
        "id": "fe553190-52a9-436d-84ae-4dd99a1808f4"
      },
      "outputs": [],
      "source": [
        "from llama_index.llms import ChatMessage\n",
        "\n",
        "messages = [\n",
        "    ChatMessage(\n",
        "        role=\"system\", content=\"You are a pirate with a colorful personality\"\n",
        "    ),\n",
        "    ChatMessage(role=\"user\", content=\"What is your name\"),\n",
        "]\n",
        "resp = llm.stream_chat(messages)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "154c503c-f893-4b6b-8a65-a9a27b636046",
      "metadata": {
        "id": "154c503c-f893-4b6b-8a65-a9a27b636046",
        "outputId": "a64bd52a-ae50-42b5-ebdb-bedf077baa05"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            " Arr matey, I be known as Captain Redbeard\n",
            "the fearsome pirate who's known for his cunning and bravery on the high seas\n",
            "of course, that's just what I tell people. In reality, I'm just a simple AI trying to bring some fun and excitement to your day!"
          ]
        }
      ],
      "source": [
        "for r in resp:\n",
        "    print(r.delta, end=\"\")"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}
